<html>
<head><meta charset="utf-8"><title>Passing large arrays by move in const · t-compiler/const-eval · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/146212-t-compiler/const-eval/index.html">t-compiler/const-eval</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/146212-t-compiler/const-eval/topic/Passing.20large.20arrays.20by.20move.20in.20const.html">Passing large arrays by move in const</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="219162255"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/146212-t-compiler/const-eval/topic/Passing%20large%20arrays%20by%20move%20in%20const/near/219162255" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Thom Chiovoloni <a href="https://rust-lang.github.io/zulip_archive/stream/146212-t-compiler/const-eval/topic/Passing.20large.20arrays.20by.20move.20in.20const.html#219162255">(Dec 08 2020 at 01:46)</a>:</h4>
<p>This is a weird question that certainly indicates I'm abusing the const evaluator, but... Lets say I know for certain this code is being const evaluated:</p>
<ol>
<li>
<p>Should I worry <em>at all</em> (in terms of compile time, I mean) about moves of large things like <code>[u8; N]</code> where (N is idk, a few megs) in arguments/returns? I'm hoping not... no moves are taking place under the hood I'd assume. (I could find a way to avoid it if needed (it's to work around the lack of <code>&amp;mut [u8]</code> but support for <code>mut [u8; N]</code>)).</p>
</li>
<li>
<p>Related: is there a risk of "const stack overflow" if I allocate a bunch of buffers like this on the stack? I'm assuming no, since there's not actually a stack.</p>
</li>
</ol>



<a name="219163581"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/146212-t-compiler/const-eval/topic/Passing%20large%20arrays%20by%20move%20in%20const/near/219163581" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> oliver <a href="https://rust-lang.github.io/zulip_archive/stream/146212-t-compiler/const-eval/topic/Passing.20large.20arrays.20by.20move.20in.20const.html#219163581">(Dec 08 2020 at 02:09)</a>:</h4>
<p>I only just read this and thought is was interesting: <a href="https://rust-analyzer.github.io/blog/2020/12/04/measuring-memory-usage-in-rust.html">https://rust-analyzer.github.io/blog/2020/12/04/measuring-memory-usage-in-rust.html</a></p>



<a name="219163637"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/146212-t-compiler/const-eval/topic/Passing%20large%20arrays%20by%20move%20in%20const/near/219163637" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> oliver <a href="https://rust-lang.github.io/zulip_archive/stream/146212-t-compiler/const-eval/topic/Passing.20large.20arrays.20by.20move.20in.20const.html#219163637">(Dec 08 2020 at 02:10)</a>:</h4>
<p>though it doesn't answer your question</p>



<a name="219188050"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/146212-t-compiler/const-eval/topic/Passing%20large%20arrays%20by%20move%20in%20const/near/219188050" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> oli <a href="https://rust-lang.github.io/zulip_archive/stream/146212-t-compiler/const-eval/topic/Passing.20large.20arrays.20by.20move.20in.20const.html#219188050">(Dec 08 2020 at 09:46)</a>:</h4>
<p>about 1., yes, we haven't optimized this too well, so there will be memcpys happening. If you come up with a small example that scales badly, don't hesitate to open an issue, we'll have a look at optimizing it. The main problem with optimizing things in the const evaluator is that we first need examples that are problematic</p>



<a name="219188117"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/146212-t-compiler/const-eval/topic/Passing%20large%20arrays%20by%20move%20in%20const/near/219188117" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> oli <a href="https://rust-lang.github.io/zulip_archive/stream/146212-t-compiler/const-eval/topic/Passing.20large.20arrays.20by.20move.20in.20const.html#219188117">(Dec 08 2020 at 09:46)</a>:</h4>
<p>there is no risk of a stack overflow in const eval. Everything is on the heap. You can run out of heap space easily though... we're working on fixing that, but the fix is just that you'll get a nice OOM error message, not that it'll work</p>



<a name="219249989"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/146212-t-compiler/const-eval/topic/Passing%20large%20arrays%20by%20move%20in%20const/near/219249989" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Thom Chiovoloni <a href="https://rust-lang.github.io/zulip_archive/stream/146212-t-compiler/const-eval/topic/Passing.20large.20arrays.20by.20move.20in.20const.html#219249989">(Dec 08 2020 at 19:22)</a>:</h4>
<p>cool. the motivation here was some code that compresses a &amp;[u8] at compile time (using a custom lz77-alike with no entropy coding). I'm trying to figure out how much I can put in a min_const_generics-<code>const fn</code> vs how much needs to be in a macro.</p>
<p>in the past I wrote <a href="https://play.rust-lang.org/?version=stable&amp;mode=release&amp;edition=2018&amp;gist=492705dbb64ae00b7ad29322f7abf956">https://play.rust-lang.org/?version=stable&amp;mode=release&amp;edition=2018&amp;gist=492705dbb64ae00b7ad29322f7abf956</a> which converts utf8 to utf16 at compile time (someting like this code ended up being used in the <code>fruity</code> crate I think).</p>
<p>It performs admirably well actually, even for big-ish files (like in the example). but it also avoids needing to do anything like this (and has some hacks to avoid const evaluator work (like transmuting <code>&amp;[T; N]</code> to <code>&amp;[T; M]</code> where M is known to be &lt;= N, rather than making a <code>[T; M]</code> and manually copying the data over in a loop)</p>



<a name="219250466"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/146212-t-compiler/const-eval/topic/Passing%20large%20arrays%20by%20move%20in%20const/near/219250466" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Thom Chiovoloni <a href="https://rust-lang.github.io/zulip_archive/stream/146212-t-compiler/const-eval/topic/Passing.20large.20arrays.20by.20move.20in.20const.html#219250466">(Dec 08 2020 at 19:27)</a>:</h4>
<p>I think this kind of thing is probably not actually great unless the const evaluator gets really fast, but for quick and dirty one-offs it could be very nice.</p>



<a name="219265693"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/146212-t-compiler/const-eval/topic/Passing%20large%20arrays%20by%20move%20in%20const/near/219265693" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Thom Chiovoloni <a href="https://rust-lang.github.io/zulip_archive/stream/146212-t-compiler/const-eval/topic/Passing.20large.20arrays.20by.20move.20in.20const.html#219265693">(Dec 08 2020 at 21:32)</a>:</h4>
<blockquote>
<p>1., yes, we haven't optimized this too well, so there will be memcpys happening</p>
</blockquote>
<p>Do you think it will be possible to remove them in the future? or should I assume stuff like that will always be costly. (it's easy enough to do the work in a macro rather than a const fn)</p>



<a name="219308093"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/146212-t-compiler/const-eval/topic/Passing%20large%20arrays%20by%20move%20in%20const/near/219308093" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> oli <a href="https://rust-lang.github.io/zulip_archive/stream/146212-t-compiler/const-eval/topic/Passing.20large.20arrays.20by.20move.20in.20const.html#219308093">(Dec 09 2020 at 08:26)</a>:</h4>
<p>We'll definitely be able to do improvements here</p>



<a name="219308121"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/146212-t-compiler/const-eval/topic/Passing%20large%20arrays%20by%20move%20in%20const/near/219308121" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> oli <a href="https://rust-lang.github.io/zulip_archive/stream/146212-t-compiler/const-eval/topic/Passing.20large.20arrays.20by.20move.20in.20const.html#219308121">(Dec 09 2020 at 08:27)</a>:</h4>
<p>one thing I always wanted to do is model moves as actual moves, this will save us lots of memory copies, since we can reuse the underlying memory</p>



<a name="219772722"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/146212-t-compiler/const-eval/topic/Passing%20large%20arrays%20by%20move%20in%20const/near/219772722" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/146212-t-compiler/const-eval/topic/Passing.20large.20arrays.20by.20move.20in.20const.html#219772722">(Dec 13 2020 at 18:15)</a>:</h4>
<p>We just have to make sure that this optimization cannot make the interpreter miss any UB</p>



<a name="219828730"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/146212-t-compiler/const-eval/topic/Passing%20large%20arrays%20by%20move%20in%20const/near/219828730" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> oli <a href="https://rust-lang.github.io/zulip_archive/stream/146212-t-compiler/const-eval/topic/Passing.20large.20arrays.20by.20move.20in.20const.html#219828730">(Dec 14 2020 at 11:34)</a>:</h4>
<p>yea, we do have to generate a new <code>AllocId</code>, but we can reuse the <code>Allocation</code></p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>